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PREFACE 


This bulletin is a report of one phase of a larger in- 
vestigation relating to the study habits of school children. 
The types of questions asked by teachers of their students 
are important both because of the mental processes which 
occur in answering them and because an intimate relation 
exists between the questions asked and the detailed objectives 
toward which the students work. 

Mr. Carter was consulted in the preparation of the 
questionnaire. He rendered valuable assistance in deciding 
upon the tentative list of types of thought questions. Later, 
advantage was taken of his presence at the University of IIli- 
nois during the Summer Session of 1922 to secure assistance in 
the preparation of this report. He is largely responsible for 
its general organization. 


WatteR S. Monrok, Director. 
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The Use of Different Types of Thought Questions 
in Secondary Schools and: Their Relative 
Difficulty for Students 


Recent emphasis upon questioning in instruction. The first 
quantitative study of the questions which teachers use in the class- 
room was made by Miss Romiett Stevens.t In this analysis of 
questions a distinction was made between those that required memory 
only and those that required thought. Miss Stevens’ discussion has 
been widely quoted and has been very influential in causing teachers 
to give more attention to the types of questions asked of their students. 
It is altogether likely that this investigation is primarily responsible 
for a material increase in the percent of thought questions which are 
asked in our schools today. Many of the most commonly used texts 
on methods of teaching devote separate space to the topic of question- 
ing. Its importance has been especially emphasized by Charters,’ 
Strayer,? Parker,* and Colvin.°® 

Analysis of thought questions. An examination of typical 
thought questions from the standpoint of the mental processes which 
they require in answering reveals certain significant differences. Some 
questions require the student to “compare” two or more ideas; others 
ask that he “‘summarize;”’ still others demand that he “give reasons 
why.” The probable mental processes occurring in the reflective 
thinking called for rather than the form of the question or the word- 
ing of the answer have been made the basis of these types of thought 


1Stevens, Romiett. “The question as a measure of efficiency in instruction,” 
Teachers College Contributions to Education No. 48. New York: Teachers College, 
Columbia University, 1912. 

2Charters, W. W. Methods of Teaching. Chicago: Rowe Peterson and Company, 
1912, p. 444. (Chapter XVII). 

8Strayer, G. D. A Brief Course in the Teaching Process. New York: Macmillan 
Company, 1912, p. 315. (Chapter XI) 

4Parker, S.C. Methods of Teaching in High Schools. Boston: Ginn and Company, 
1915, p. XXV, 529. (Chapter XX) 

Colvin, S.S. Introduction to High School Teaching. New York: Macmillan 
Company, 1917, p. XXI, 451. (Chapter XV) 
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questions. Altho there is doubtless considerable overlapping in the 
types of mental processes which usually occur in answering such 
questions, there appears to be sufficient differences to justify the re- 
cognition of a number of types of questions. In this investigation 
the following tentative list of types of thought questions was used. 
A more refined analysis would yield probably some additional types, 
but, on the other hand, for practical purposes it is possible that some 
combinations of types might profitably be made. It is recognized 
that the mental processes of different students doubtless vary in 
answering the same question. In fact a thought question for one stu- 
dent may bea memory question for another student, or even for the 
same student on the following day. 


1. Selective recall—basis given. 

Name the presidents of the United States who had been in military life before 
their election, 

What do New Zealand and Australia sell in Europe that may interfere with our 
market? 


2. Evaluating recall—basis given. 
Which do you consider the three most important American inventions in the 
nineteenth century from the standpoint of expansion and growth of transportation? 
Name the three statesmen who have had the greatest influence on economic 
legislation in the United States. 


3. Comparison of two things—on a single designated basis. 
Compare Eliot and Thackeray in ability in character delineation. 
Compare the armies of the North and South in the Civil War as to leadership. 


4. Comparison of two things—in general. 


Compare the early settlers of the Massachusetts Colony with those of the Virginia 
Colony. 


Contrast the life of Silas Marner in Raveloe with his life in Lantern Yard. 
5. Decision—for or against. 

Whom do you admire more, Washington or Lincoln? 

In which in your opinion can you do better, oral or written examinations? 
6. Causes or effects. 


Why has the Senate become a much more powerful body than the House of 
Representatives? 


What caused Silas Marner to change from what he was in Lantern Yard to what 
he was in Raveloe? 


7. Explanation of the use or exact meaning of some phrase or state- 
ment in a passage. 


8. Summary of some unit of the text or of some article read. 
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9. Analysis. (The word itself is seldom involved in the question.) 
What characteristics of Silas Marner make you understand why Raveloe people 
were suspicious of him? 
Mention several qualities of leadership. 
10. Statement of relationships. 
Why is a knowledge of Botany helpful in studying agriculture? 
Tell the relation of exercise to good health. 
11. Illustrations or examples (your own) of principles in science, con- 
struction in language, etc. 
12. Classification. (Usually the converse of No. 11). 


What is the principle involved here? What is the construction? To what class 
or genus does this individual belong? 


13. Application of rules or principles in new situations. 
14. Discussion. 

Discuss the Monroe Doctrine. 

Discuss early American Literature. 

15. Statement of aim—author’s purpose in his selection or organiza- 
tion of material. 

What was the purpose of introducing this incident? 
Why did he discuss this before that? 

16. Criticism—as to the adequacy, correctness, or relevancy of a 
printed statement, or a classmate’s answer to a question on the 
lesson. 

17. Outline. 

18. Reorganization of facts. (A good type of review question to give 
training in organization.) 

The student is asked for reports where facts from different organizations are 
arranged on an entirely new basis. 

19. Formulation of new questions—Problems and questions raised. 


What question came to your mind? 
What else must be known in order to understand the matter under consideration? 


20. New methods of procedure. 

Suggest a plan for proving the truth or falsity of some hypothesis. 

How would you change the plot in order to produce a certain different effect? 

Relation to educational objectives of types of questions asked 
by teachers. Incidentally it may be noted that the types of questions 
used both for stimulating and directing the mental activity of the 
learner and for measuring the results of teaching reflect in a subtle 
way the educational objectives of a teacher. The objectives thus 
indicated may not agree with those stated by the teacher but they 
nevertheless are an index of the objectives toward which the students 
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direct their efforts. In studying, students tend to prepare to answer 
the kind of questions which they think the teacher will ask. It 1s 
altogether likely that the study objectives of students are influenced 
more by the kind of questions asked than by direct statements of 
aims in the course. 

Purpose and method of this investigation. The purpose of this 
study, which is Sub-project II of our investigation of the study 
habits of high school pupils, was announced as follows: “To deter- 
mine the extent of the use of different types of thought questions in 
actual school practise and the relative difficulty of these types for 
students.” The method employed was to submit a questionnaire to a 
large number of teachers in the seventh and eighth grades and in the 
high school. In this questionnaire the list of the twenty types of 
thought questions given on pages 6-7 was reproduced. The teachers 
were urged to study these types carefully before answering any of the 
questions. It was pointed out that an attempt to classify a question 
according tothis plan might lead to the conclusion that it is a combina- 
tion of two or three types. However, in such a case, it is probably 
true that part of the work of answering has been done by the author 
in the textbook, leaving to the student only that phase of the question 
which would definitely classify it as belonging to one of the above 
types. 

Plan of summarizing data. Usable replies were received from 199 
teachers representing almost every school subject and all parts of the 
state. A few of these replies did not give answers to one or more of 
the questions but they were included in the tabulation for the other 
questions. By subject the number of replies were as follows: English, . 
41; History, 48; Science, 41; Mathematics, 31; Foreign Language, 12; 
Geography, 11; Commercial subjects, 6; Agriculture, 4. Taking only 
the replies from teachers of English, History, and Science there were 
26 for grades VII and VIII combined; 34 for grades [IX and X; and 
45 for grades XI and XII. It was thought advisable to summarize 
separately the replies for English, History and Science. Those re- 
ceived from teachers of all other subjects were grouped together. 
The replies to Questions I and IV obviously do not lend themselves 
to statistical treatment. The data yielded by the other three ques- 
tions are summarized in the following tables. 

Limitations of replies to the questionnaire. Several teachers 
took occasion to mention difficulties which they experienced in filling 
out the questionnaire blank. That some difficulties were encountered 
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is not surprising because the concept of different types of thought 
questions was undoubtedly new to most, if not all, of the teachers. 
The limitations of space prevented an extensive description of each 
type. Undoubtedly teachers differed in their interpretations of the 
various types. Furthermore, in answering Question II they were 
asked not only to differentiate between thought questions and mem- 
ory questions but also to classify the thought questions under the 
various types. In doing this, they had to rely upon their memory of 
the questions they had asked. For these reasons the replies to Ques- 
tion II must be thought of as giving only a very rough indication of 
the practise of teachers with reference to the types of questions which 
they asked. A few teachers reported that their replies were based on a 
careful analysis of lists of questions which have been used and pre- 
served. If these lists were reasonably complete their replies should be 
more accurate than those received from teachers who relied upon their 
memory of the questions they had asked. 

It may be pointed out that other methods of ascertaining the 
relative frequency of use of different types of thought questions would 
not eliminate all difficulties. For example, if a trained investigator 
should visit the classes of 199 teachers in various parts of the state a 
sufficient number of times to obtain a fair sample of the habitual 
practise and should try to make a record of all types of questions 
asked he would encounter difficulties in definitely classifying them. 
It is not possible to judge accurately of the mental processes stimu- 
lated by a question unless one is acquainted with the previous ex- 
periences of the pupils in the field of this question. This has been 
referred to already in pointing out that what constitutes a thought 
question for one pupil may be merely a memory question for another. 
The analysis of stenographic reports of lessons would present certain 
difficulties for the same reason. The teacher knows better than any 
one else what acquaintance a student has with a topic and what 
the textbook states about it. For this reason he has the advantage of 
a casual observer. 

Additional types of questions mentioned. Question I was asked 
‘n order to obtain suggestions for supplementing the list of type 
questions for further study in this field and also to give each teacher 
an opportunity to report fully his own practise in the use of different 
types of thought questions in case he did not consider the list ade- 
quate. The replies to the questionnaire show that the number of 
teachers who did not report the use of all of the types.given in the list | 
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. In the left hand margin make an “‘X”’ opposite each of the five types for which 


students answers are least likely to be satisfactory. (See directions, p. 11) 


. Choose three of the types you marked with an ““X.” State for each some of the 


students’ most-common faults in procedure in answering it. Mention faults of 
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omisson as well as faults of commission. Mention only faults peculiar to this 
type of question. Use other side of this sheet if necessary. (See directions below.) 


V. If we consider all unsatisfactory answers made by students in school work, what 
percent of them, in your opinion, are due primarily to bad habits of procedure 
in answering questions? Give separate replies for memory questions.......... %G 
and thought questions............ % 


The following detailed directions for answering the questionnaire 
were given: 

Before answering any of the questions, decide what subject and what grade 
- (any from VII to XII) you will have in mind in all of your replies. Answer the questions 
with reference to only ove school subject. Fill in the blanks at the top of the first page 
of the questionnaire. 


Question I. Examine the list of types given above to see if there are not some 
other types of thought questions that you use to a considerable extent. Altho the given 
list may seem long and inclusive at first, it is not complete. 


Question II. Before you put down your estimates for Question II, you should 
decide whatis your proportion of thought questions and of pure memory questions. 
This question calls for a further analysis of your thought questions. It may be advis- 
able to make rough estimates for all before you fill in the data on the questionnaire. 
After you have the differences between the types in mind, make the best estimate you 
can, even tho you may not be very sure in many cases. You are merely declaring your 
best judgment, not guaranteeing that it is infallible. We want your opinion of what 
your practise is, not of what you think it ought to be. 


Question III. In Question III, we are concerned with the process of answering 
so we must assume that the student has the information necessary for a satisfactory 
answer if he will only use it as the type of the question requires. The common starting 
point for an informal diagnosis of a student’s study faults is in his unsatisfactory 
answers. An unsatisfactory answer may indicate lack of information or certain faulty 
habits in answering questions. There are many cases in which teachers find out by 
other means that the student has enough information to give a satisfactory answer 
but has failed because he did not take the necessary steps and precautions in replying 
to the question. When some students correct examination papers returned to them, 
they are heard to say, “I knew that, but I didn’t think of it,” or “T knew that but I 
didn’t know that was what the question meant.” Some of the faults in procedure are 
common to all kinds of questions; others are peculiar to particular types. In Question 
III, you are asked to decide on the difficulty of the types for students. In other words, 
in which type do they have the worst habits of procedure? 

Question IV. It will be helpful in answering Question IV to think of some par- 
ticular questions of the type under consideration and then express the students’ faults 


in fairly general terms. 

Question V. As long as the unsatisfactory answer is a resultant of poor methods 
ng questions, we must correct the latter or 
methods of study are needed. Itisimportant 
hers of the percent of failures (in answers) 


of preparation and poor habits of answeri 
take it into consideration in inferring what 
to get an estimate from a large number of teac 
that are due primarily to bad habits of procedure in answering questions, 
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is very much greater than the number mentioning additional types. 
Only thirty-nine teachers (19.6 percent) indicated that they used all 
twenty types of questions. Thirteen teachers supplemented the list, 
each writing in from one to three additional types. These teachers 
were distributed among the different subjects as follows: seven in 
History, three in English, two in Science, and one in Mathematics. 
Only four of those suggesting additional types of questions indicated 
that they used all twenty of the types given in the questionnaire list. 

Some of the additional types of questions suggested are clearly 
included in the list given in the questionnaire. The following are 
typical: (A) “Connections between historical events,’ Type 4 or 6; 
(B) “Cause and results,” included in Type 6; (C) “Application to the 
present time,” included in Type 13; (D) “Determining the significant 
word in a statement,” included in Type 19. Other suggested types are 
not so clearly included but several of them under a fairly loose inter- 
pretation may be placed with the types listed. Illustrations of these 
are: (A) “Imagining the results if conditions were different,” a 
special case of Type 6; (B) “Questions on appreciation such as, which 
do you like best?” a fairly common form of Type 2; (C) “Identify a 
known character in a scene in the story when his name is not mention- 
ed by the author,” a special case of Type 12; (D) ‘““What should you 
judge from these facts?” This last is broad enough to cover many 
types. Frequently, it would fall under Type 6. There were a few 
suggestive questions which are not so easily classified under the 20 
types given. Some of these are: (A) “Trace the development,” 
mentioned by two teachers; (B) “Estimate the importance of”; 
and (C) “Why is this statement true?” a very common question in 
geometry. By a very liberal interpretation these might be put under 
Types 9, 2, and 6 respectively but it is likely that many teachers 
would consider them sufficiently different to justify naming them as 
additional types. 

The answers to the first question indicate that the list of types 
is reasonably complete. For practical purposes it is probably better 
to give a rather loose interpretation in classifying special cases than 
to try to extend the list. Largely for this reason the authors decided 
to limit the summary of the replies to the other parts of the ques- 
tionnaire to the original twenty types. 


Frequency of use of different types of questions. Question II of 
the questionnaire was asked in order to secure answers for the 
following: 
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1. Which types of questions are most commonly used by teachers 
in classroom instruction? 

2. Which types of questions are most characteristic of the in- 
struction in different subjects? 

3. Do teachers in the more advanced grades tend to ask different 
types of questions from those in the lower grades? 


Table I gives a detailed summary of the replies by forty-eight 
teachers of History. This table shows that four of these teachers did 
not report any use of questions requiring selective recall (Type 1); 
one teacher reported that this type of question formed 1 percent of 
all his questions; five teachers indicated that this type made up 2 
percent of all the questions which they asked; eleven named 5 percent 
as an index of its frequency; eight teachers considered that 15 per- 
cent of their questions were of this type; and another eight teachers 
indicated that more than 15 percent of their questions called for 
selective recall. The median practise is 7.5 percent. 

The outstanding characteristic of the table is the variability of 
practise which it indicates. In the case of each type of question there 
were one or more teachers who gave no indication of its use. For 
several of the types the number of teachers not indicating any use is 
surprisingly large. Furthermore, there is no type of question which 
did not receive an indication of at least 5 percent by one teacher. 
Most of the types received an indication of 10 percent or more. Thus, 
if we may assume that the replies to this portion of the questionnaire 
are a reasonably true indication of practise, it is clear that teachers 
of History vary widely in the types of thought questions which they 
ask of their pupils and hence necessarily vary widely in the detailed 
objectives which their pupils strive to attain. 

Similar tabulations were made for English, Science and other 
subjects combined. The median frequencies for the four groups of 
subjects are given in Table II. If we consider only English, History 
and Science, students are most commonly required to answer the 
following types of thought questions: cause and effect, Type 6; 
selective recall, Type 1; discussion, Type 14; and evaluating recall, 
Type 2. The types of questions which are asked least frequently are: 
formulation of new questions, Type 19; new methods of procedure, 
Type 20; and reorganization of facts, Type 18. 

Different subjects require different mental processes. An ex- 
amination of Table II reveals that the frequency of use of some of the 
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TABLE II. MEDIAN FREQUENCY OF USE OF DIFFERENT TYPES OF 
THOUGHT QUESTIONS IN ENGLISH, HISTORY, SCIENCE AND 
OTHER SUBJECTS 
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types of thought questions varies in the different subjects. For 
example, Type 7 “Explanation of the use or exact meaning of some 
phrase or statement in a passage” is the most frequently used type 
of question in English but it is eleventh in frequency of use in His- 
tory. Asking students to give an illustration or example, Type 11, 
is next to the most frequent type of question in Science but it is sel- 
dom used in History. Type 10, which asks the student to state a re- 
lationship, is frequently used in Science but is infrequently asked of 
students in English. Summaries and outlines are required more 
frequently in History and English than in the other subjects. Thus, 
if our assumption that different types of questions require different 
mental processes is valid, we have here evidence that different types 
of mental processes are required of students in the different subjects. 
Hence, we should expect to find that the problem of directing the 
learning of students is different in different subjects. 


[15] 


TABLE III. PERCENT OF TEACHERS REPORTING NO USE OF CERTAIN 
TYPES OF THOUGHT QUESTIONS 


Grades Grades Grades 
Types of Questions VILVI | IX-X XI-XII 
{eSeléctive recalloy. sma s cietharccste.siier o'er 3 18 11 
DeEvalnatine Tecallojeneme seen steers 6 29 13 
Ga Gomparison—-Sin gies oisese sicteleie state 18 9 18 
4..Gomparison—generals sensei) ae 9 18 13 
5. Decision—for or against............. 21 OL 27 
Om Casesotiemects: pit ere iit 9 6 4 
HMExplanation wear see «nase aerials ae 23 26 18 
SIESUINIMALVoceey recess eee weno eee 6 23 22 
OMAnalysisnce Seon jotde. cots sete e ick 21 23 29 
10 BRelationshipssesertc sn ete oe 35 18 20 
tie itilustration orexamples....... 0-4-1 41 26 16 
if De Classification wri: aiasse ean eer 50 29 24 
1S Application safc acheter athe cece ace obs 46 35 22 
TA DISCUSSIONS soot satya isons fot istias ete. ose 9 15 13 
(SoeStatemenG ofvaim are nein see ater 63 46 42 
LGee Criticism ees see reece eet eeu 26 25 24 
LO utlinever rete Mots ation etka ce ees 26 21 31 
1S mReoreanrzationiom acts aera eerie 35 41 42 
19. Formulation of new questions.,....... 68 44 44 
QOMINEwrmethodsssaccacm aoe seer 44 53 32 


Variations in the use of type questions in different school grades. 
The replies of 113 teachers have been summarized in Table III on the 
basis of the school grades in which they were teaching. The subjects 
involved are English, History, Science and Geography. Instead of 
giving the median frequency of use, the percent of teachers indicating 
no use of the given type of question has been calculated. Thus, 3 
percent of the teachers in the seventh and eighth grades reported no 
use of selective recall. In the ninth and tenth grades 18 percent of the 
teachers did not use this type and in the eleventh and twelfth grades, 
11 percent. The general impression prevails that students in ad- 
vanced classes are asked more difficult types of questions than stu- 
dents in the lower grades. A comparison of the data for the three 
grade groups shows that the differences are not very great. In gen- 
eral, it appears that the variations in the use of thought questions are 
greater for different subjects than for the different grades. 


The relative difficulty of the different types of questions for 
students. In Question III teachers were asked to designate the five 
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TABLE IV. TYPES OF QUESTIONS FOR WHICH STUDENTS’ ANSWERS 
ARE LEAST SATISFACTORY 


Types of Questions 


1. Selective recall........ 5 10 8 14 37 15 
2. Evaluating recall...... 12 12 11 10 45 8 
3. Comparison—single... . 6 7 9 5 27 19 
4, Comparison—general. . 12 12 9 14 47 6 
5. Decision—for or against 1 9 42 6 18 20 
6. Causes or effects....... 17 17 19 24 77 1 
Heerxplanations sj. =" 15 12 15 25 67 3 
Ch Shiels Reoeeanae oo cen 10 14 9 14 47 6 
Om@Analysisce ue -s sess 13 15 11 21 60 5 
10. Relationships......... 5 17 10 20 42 10 
11. Illustration or examples 5 4 10 14 33 17 
OPE Glass fice Ons e ese: 5 4 g) 16 34 16 
tsa Application... 1: 5-2. 16 12 18 29 77 1 
14 wDISCUSSION Ac-6)56 a>: 10 14 5 10 39 13 
15. Statement of aim...... 15 12) 5 10 42 10 
Gra Griticismi «creat « 22 << 8 14 7 10 39 13 
Aer OUtiNes se sass > + 6 12 6 9 33 17 
18. Reorganization of facts.| 12 19 13 19 63 4 
19. Formulation of new 
GUMESHODS emlecis as - a 9 6 8 18 41 12 
20. New methods......... 6 7 7 24 44 9 


types of questions in which “students’ answers were least likely to be 
satisfactory.” A summary of their replies is given in Table IV. ‘The 
first line of this table should read as follows: Questions calling for 
selective recall were named among the five most difficult types by five 
teachers of English, ten teachers of History, eight teachers of Science, 
and fourteen teachers of other subjects. This type of question ranks 
fifteenth in the frequency of mention among the five most difficult 
types. The two types most frequently mentioned were Type 6, 
causes and effects, and Type 13, application. Both of these were 
mentioned by seventy-seven teachers or 38.7 percent. Altho we may 
say that in the judgments of those answering the questionnaire these 
two types are the most difficult for students, it should be noted that 
over 60 percent of the teachers did not list them among the five most 
difficult types. 

Here, as in other tables, the variation in the replies of teachers 
is very extreme. Even when we consider the teachers of a single 
subject we find marked variations in their judgments concerning 
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the difficulty of the different types of questions. No type of question 
is so easy that it is not included among the most difficult five by 
several teachers. This variation in judgment is probably due largely 
to the fact that teachers have not recognized the distinction in types 
of questions which are made here and, furthermore, they have not 
analyzed the responses of their students in order to ascertain the 
nature of the difficulties which the students encounter in answering 
the questions. 

Common faults of procedure in answering different types of 
questions. In Question IV the teachers were asked to select three of 
the types which they considered most difficult and to state for each 
of these some of the students’ most common faults in procedure in 
answering them. An examination of the replies to this question 
suggests that a considerable number of teachers are satisfied with 
telling students that the answer is wrong, or what the answer should 
have been, and that they fail to give much thought to the students’ 
faults of procedure in answering the question. It may be pointed out 
that questions are only a means to an end and this end is the education 
of the student. The answer is in itself relatively unimport- 
ant. The question fulfils its function only when it stimulates 
educative processes in the mind of the student. If the mental processes 
which the question initiates are not educative the asking of the ques- 
tion has been largely futile. Hence, it becomes highly important for 
the teacher to give attention to the procedure which the student em- 
ploys in answering questions in order that the faults of procedure may 
be corrected so that the student’s mental processes will become most 
effective in his education. 

Of the 199 teachers from whom replies to the questionnaire were re- 
ceived, twenty-three did not attempt to answer Question IV at all and 
twenty-four others discussed all three types of questions together or 
stopped after discussing only one or two. Furthermore, a number of 
other teachers gave answers which show that they failed to grasp the 
significance of this question. In substance they said that the difficulty 
in students’ procedure in answering a question is inability to do what 
is called for. Forexample, one said that the fault in answering Type 2, 
evaluating recall, is that “pupils are not always able to evaluate.” 
Another said that in questions on aim “they seem unable to see any 
motive behind the statement of theauthor.” Still another said that 
the fault in Type 6, causes and effects, is that “they are unable to see 
causes and effects.” Still others mentioned objective faults in the 
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answers rather than faults in procedure. For example, one teacher 
mentioned that “minor details are included while leading points 
are omitted.” Another stated that “applications are not as good as 
could be expected.” Still another teacher simply stated that “the 
answers are inaccurate.” General faults, suchas “lack of practise in 
answering questions,” “carelessness,” “lack of concentration,” “pupils 
do not know how to study,” and so on were mentioned by one or more 
teachers. 

It may be pointed out that teachers who fail to become definitely 
conscious of the difficulties which their students encounter will be 
unable to give them much definite constructive assistance in this 
phase of their learning. Probably the most significant conclusion to 
be drawn from the answers to this part of the questionnaire is that 
many teachers are failing to give attention to the procedure which 
students use in answering questions. They appear to be concerned 
largely with the accuracy of the answer and when it is wrong they 
fail to seek the cause in the procedure which the student has used. 

Several of the faults which appear to be suggestive are given 
below. This list is not a complete statement of the faults of students 
in answering questions but should prove helpful to teachers who are 
desirous of ascertaining the reasons why students fail to answer 
questions satisfactorily. 


eet XS 


Type 2. 

The pupil fails to see real basis on which evaluation should be made. 
Type 3. 

A student seldom stays “‘on a single designated basis.” 
Type 3 or 4. 


The pupil begins to answer the question before he has thought out what points 
should be included in the comparison. He often includes minor comparisons and omits 
the important points. 

Unless there is a single designated basis for comparison, pupils fail to see all of the 
possibilities of comparison. The slothful pupil is content with one or two obvious com- 


parisons. . 
The pupil can state some information about each thing compared, but omits 


the comparative connection between the two things. The pupil gives comments on 
one thing and omits the discussion on the other thing, showing how the two are alike 
comparatively. 

In making comparisons, students usually give characteristics of things to be 
compared, or describe them separately but fail to make a complete comparison. 
Type 4. 

In comparing two things in general the worst fault is in the students’ forgetting 
some of the important points to be compared. Often enough suggestions must be 
given so that the question finally should be placed under Type 3. 
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Pupils frequently fail to distinguish the important from the trival. The pupil’s 
prejudices—likes and dislikes—influence the comparison. 


Type 6. 

Students confuse cause and effect. Method requires real thinking and reasoning— 
the hardest thing any student can be asked to do. 

Students often know a statement is true but on account of lack of thoroness can 
not give causes or reasons. 


Type 6 or 7. 
Pupils, when asked, “Why?” seem to forget all reasoning and usually state the 
first thing that comes to mind and fail also to go far enough in explanation. 


Type 7. 

Pupils have difficulty in finding words to express the thought without repeating 
word for word parts of the passage to be explained. The chief difficulty seems to be the 
limitations of their vocabulary. 

The trouble here is that pupils seem to have failed to study the statement under 
question sufficiently and so have quite an indefinite notion of the words used. And too, 
even if they succeed in getting a fairly good understanding of the text read, they are 
too careless of the choice of their own words in reproducing the meaning exactly. 
They are too hasty in the study of the passage and have not yet been trained to go 
slowly and get facts one at a time. 


Type 8. 

A discussion, like a summary often lacks point. The student fails to organize 
material and grasp essentials. 

Students fail to properly organize data mentally before expressing them. 


Type 9. 


Here pupils are often prone to feel satisfied if only a few of the factors or qualities 
involved in the analysis are learned and reproduced. Many times but one factor will 
be offered when to completely analyze the problem will require several. Being allowed 
to “get off” with a minimum of thinking is one of the chief causes of poor analysis. 


Type 10. 

Pupils often fail to get the facts correlated before they attempt to state the exist- 
ing relationships. 

Pupils do not stop to analyze in order to see the relationship but answer without 
giving much thought, 

Pupils answer various separate facts instead of showing relationship. These facts 
are usually true and have a bearing upon the question but the actual relationship is 
usually missing in the answer. That is, the pupil does not seem to see how a change in 
one fact would influence some other related fact. 


Type 11. 


Illustrations or examples. This is not so difficult after the student has had some 
practise, but at first he always tries to find his illustrations in some field remote from 
his own. It takes him a while to realize that scores of illustrations come daily under his 
observation—if he observes. 
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Type 12. 

In classifying, the pupils make their answers too readily without first trying one 
class then another or without thinking whether the one they choose actually does fit 
better and why. 

The student’s failure in Type 12 is due mainly to his failure in Type 7. He fails 
to decide correctly upon a construction because he has not thought out carefully the 
exact meaning of the sentence. This in turn is due to mental laziness or carelessness, 


Type 14. 


He discusses each phase at random and separately without relating it to others. 

A discussion like a summary often lacks point. The student fails to organize 
material and grasp essentials. 

Students fail to properly organize data mentally before expressing them. 

Students do not group their facts together sufficiently to discuss them properly. 


Type 17. 

A pupil doesn’t really outline into main facts with heads under them, but makes 
long statements one after another, or puts steps in before their time. 
General. 

Pupils will not read and get a thoro knowledge of the subject-matter first. 

With Types 8, 14, and 17 pupils do not summarize or discuss or outline ina way 
to make clear to another the things under discussion, They are not logical in the ar- 
rangement of their ideas. They answer, at least 85 percent of them, only well enough 
so that one who knows already can tell that they know, but only about 15 percent 
can arrange and subordinate well enough to be clear to the unintiated. 

A common fault is careless reading of the question (or hearing it as the case 
may be). This is especially true in Algebra. Another fault is ‘saying the first thing that 
comes into mind” instead of carefully considering its application to the problem under 
discussion. In Geometry, for example, the student does not rapidly recall all theorems 
pertaining to the subject in hand, eliminate those inapplicable, and so discover the 
correct one. 

Percent of unsatisfactory answers due to faulty procedure in 
answering questions. In Question V the teachers were requested to 
indicate separately for memory questions and for thought questions 
the percent of unsatisfactory answers which, in their opinion, were 
due primarily to bad habits of procedure. A summary of their re- 
plies is given in Table V. As previously indicated it appears certain 
that a considerable number of teachers have given relatively little 
thought to the faults in the procedure used by students in answering 
questions. Consequently, the replies to this question must be con- 
sidered as representing no more than rough estimates. In the case 
of thought questions the median percent of unsatisfactory answers 
due to faulty procedure 1s approximately 50 for all of the subjects. 
In the case of memory questions faulty procedure is considered to be 
a less potent cause. There are two factors that probably tend to 
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make these estimates lower than they should be. Many teachers 
showed by their replies to Question IV that they had very little ex- 
perience in analyzing answers for the purpose of determining the cause 
of faulty answers. Naturally they could not be expected to recognize 
the real extent of such faults of procedure on the part of the members 
of their classes. Other teachers use such. a limited range in types of 
questions that they would not have a chance to observe as many 
faults in answer technique as teachers who use a greater variety of 
types of questions. 

Relation between frequency of use and difficulty. There seems 
to be very little relation between the difficulty of a type and the 
frequency of its use. Type 6, causes and effects, is the most commonly 
used type and at the same time it, together with Type 13, application, 
is considered the most difficult for students. Only one of the five 
types reported as most difficult, Type 18, reorganization of facts, 
is among the five least frequently used; only one of the five types 
reported as least difficult, Type 13, comparison on a single basis, is 
among the five most frequently used. 

Relation of frequency of use of types of questions to emphasis 
upon different types of learning. An earlier bulletin’ of the Bureau 
of Educational Research reports the relative emphasis on different 
types of textbook study in the various subjects. We now have some 
data on the question, “How does the frequency of use of types of 
questions correspond with the emphasis on the different types of text- 
book study required of students?” Some of the types of study 
recognized in the report are too broad to be tested by any one type 
of thought question. There are, however, question types that cor- 
respond rather closely with certain of the study types. 

To get a rough measure of the relation of the frequency of use of 
the different types of thought questions to the emphasis upon the 
types of learning, we may arrange the types in order of frequency of 
use and divide the list into quartiles (four equal parts). For instance, 
in English, Study Type II is in the first quartile (i. e. the three most 
frequently required study types); Study Type X is in the fourth 


1Monroe, Walter S. “Types of learning required of pupils in the seventh and 
eighth grades and in the high school.” University of Illinois Bulletin, Vol. 19, No. 15, 
Bureau of Educational Research Bulletin, No. 7, Urbana: University of Illinois, 1921, 


16. p. 
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Types oF TEXTBOOK STUDY Types OF QUESTIONS 


IJ. Summary of central ideas in the 8. Summary. 
lesson studied. 
III. Prepare an outline of principal points 17. Outline. 
and supporting details in the lesson 
arranged to show order of relative 
importance and relationship. 
VI. Discovery of collateral or illustrative | 11. Illustration or examples, 
material for topics or problem under 


discussion. \ 
VIII. Appreciation of the significance of | 7. Explanation of the use or meaning of 
each word used in a concisely ex- some phrase or statement in a passage. 


pressed statement or principle. 
IX. A clear comprehension of the essen- | 9. Analysis. 
tial conditions of a problem which is 
to be solved. 
X. Discovery of new or supplementary | 19. Formulation of new questions, 
problems related to the topic being 
studied. 


quartile (i. e. one of the least frequently required study types).! 
Table VI shows the extent of agreement in rank in frequency of use 
of some corresponding types of study and questions. The first column 
gives by number the corresponding types as listed above. The first 
number given in the columns for the separate subjects designates the 
quartile rank of the study type on the same horizontal line; the 
second number designates the rank of the corresponding type of 
question. It will be noted that the rank agrees in seven out of eighteen 
possible cases. In English the disagreement is greatest in the case of 
questions calling for analysis (first rank) and study requirements of a 
clear comprehension of the conditions of a problem (fourth rank); 
in History, the discovery of collateral or illustrative material ranks 
second in study requirements but questions calling for illustrations or 
examples rank fourth in use; in Science, the same study requirement 
ranks fourth and the corresponding question type ranks first. Thirty- 
three percent of the Physics and Chemistry teachers in the first ques- 
tionnaire mentioned Study Type X, discovery of new or supple- 
mentary problems, as a major type of study in these subjects, yet 
44 percent of the teachers of these subjects by their admission do not 
ask any questions of Type 19, formulation of new questions, to their 


1These facts are derived from Table II of the Bulletin referred to above. 
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high school juniors and seniors. It would seem from the data that 
some teachers may not be using the types of questions that would 
stimulate and test students’ achievements in some of the types of 
study they are expected to use. 


Conclusions. Probably the most significant conclusions to be 
drawn from this investigation are: (1) teachers are not sufficiently 
conscious of the types of questions which they are accustomed to ask 
and of the significance of these types, and (2) in general teachers do 
not analyze unsatisfactory answers to questions in order to ascertain 
whether such answers are due to a faulty technique on the part of 
the student. 

A number of other conclusions are worthy of mention. (1) 
Teachers of the same subject vary widely in the extent of their use of 
different types of thought questions. (2) The frequency of the use of 
a given type does not seem to depend very much on (a) the school 
grade, (b) the subject, or (c) the supposed difficulty of the type. It is 
very likely that some teachers, who expect their students to use cer- 
tain types of study, do not use the types of questions that are best 
suited to test their students’ achievements due to those particular 
types of study. (3) Teachers individually think that certain types of 
questions are more difticult for students than others. However, there 
is no very great agreement among teachers as to the relative diffi- 
culty of the various types. 
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